This invention relates generally to graphical image manipulation systems, and more particularly to a method for creating and editing electronic images of documents.
Two major classes of image editors are structured graphics, or vector-based editors, and digital paint, or raster-based editors. Structured graphics editors are suitable for editing graphic objects such as lines, curves, polygons, etc. Other types of images such as photographs, are more suitably edited in “paint” style editors that preserve the full variation and tone of the markings in terms of a two-dimensional raster of pixel intensities. Paint style image editing programs support the import and editing of raster-format electronic images. Various means are provided for selecting image regions for further manipulation such as deleting, copying, moving, rotating, and scaling. These programs are designed for editing of general photographic images, and they are limited in the degree of support they provide for the more specialized features and requirements of editing raster images of documents.
Paint style programs maintain an electronic canvas of pixel intensities. In some programs, the user is presented with a very simple usage model, which is easy to understand but offers limited functionality. In simple paint programs there is only one canvas layer. The process of selecting and moving image material causes the pixel values in one image region to replace the pixel values in a corresponding region at a different, destination location. Once such a procedure is completed, there is no notion of an image object as such that can be re-selected, to replicate the previous selection operation. This shortcoming is particularly problematic when some pixel values are treated as transparent: in this case pixels with these values do not replace pixel values in the destination location, and then two image objects can become intermingled and effectively inseparable.
Other, more complex paint-style programs, offer greater functionality but are much more difficult for users to understand and operate. In these programs multiple canvases represent different layers, where the topmost visible layer determines what is actually rendered as the visible image. Layers are dealt with explicitly by the user through complex sets of keyboard and mouse commands and auxiliary windows. Users can cause new empty layers to be created, and they can perform operations which cause regions of any source layer to be removed and copied to a different destination layer. The user maintains control over the ordering of layers. When a user wishes to move or modify an object, they must find the corresponding layer, then shift that layer's positioning over the base canvas. If the user wishes to move or modify several objects at once, then a lengthy series of steps must be undertaken to get all of the objects onto a single layer or group of layers treating the objects as a unified collection.
In both simple and complex paint programs, certain pixel color/intensities can be defined to be transparent so that pixels from layers beneath them are made visible, as is illustrated in FIG. 1. As shown in FIG. 1, when a user performs a selection gesture 110 to select a visible image object to move, the selection gesture includes not only the visible markings 120, but also surrounding background pixels 130. If the background is not rendered transparent, when the selected region is moved to location 140, it may obscure other visible markings already present in that space. In this example the obscured material is represented as 160.
In some applications, a specified pixel intensity, such as white, is predefined as being potentially a transparent value. In others the user specifies a range of pixel colors/intensities to be treated as transparent on one or more layers. In other applications, image processing operations may be applied to the image. If a high-pass filtering operation is available and applied by the user, then that could have the effect of regularizing a mottled or blotchy background of a scanned document so that potentially a small range of color/intensity values could be assigned to set the background to behave transparently.
U.S. Pat. No. 5,548,700 to Bagley et al. titled “Editing Text in an Image” teaches a data structure and procedures for subdividing a document image into smaller raster image objects which collectively become rendered to final image. However, Bagley et al. is directed to keyboard-based editing images of printed text, rather than to mouse or stylus-based editing of more general document images including handwritten scribbles and graphics.
U.S. Pat. No. 5,553,224 to Saund et al. titled “Method for Dynamically Maintaining Multiple Structural Interpretations in Graphics Systems” discloses an approach to maintaining a lattice grouping structure in curvilinear line art in which curvilinear segments with co-terminal endpoints are grouped according to their alignment and corner configurations. However, it does not pertain to arbitrary image objects, but only to curvilinear strokes.
U.S. Pat. No. 5,889,523 to Wilcox et al. titled “Method and Apparatus for Dynamically Grouping a Plurality of Graphic Objects” teaches a cluster tree for dynamically grouping a plurality of graphic objects. The cluster tree is based on a distance metric indicating a distance between a pair of graphic objects, with each level of the cluster tree defining a new cluster of graphic objects. The different cluster levels of the cluster tree containing a selected graphic object are displayable and can be modified to increase or decrease the cluster level of the cluster containing the selected graphic object.
U.S. Pat. No. 5,974,198 to Hamburg et al. titled “Adjustment Layers for Composited Image Manipulation” teaches the use of additional layers in the modification of composited images. Specifically, one or more adjustment layers are applied to an intermediate merged image, generated by compositing previous image layers, and the adjusted result is stored as a temporary image. The temporary image is then composited with the intermediate merged image. Any remaining image layers are then composited in with the intermediate merged image to generate a final merged image.
U.S. Pat. No. 6,028,583 to Hamburg titled “Compound Layers for Composited Image Manipulation” teaches a method for compositing a set of ordered image layers, in which a compound layer contains a plurality of image layers. Image layers under the compound layer are composited to generate a first intermediate image, the first intermediate image is composited with each image layer in the compound layer to generate a second intermediate, the first intermediate image is composited with the second intermediate image according to the compound layer effect to generate a third intermediate image, and the third intermediate image is composited with any remaining image layers to generate a final image.
The present invention offers a new tool for computer assisted drawing, one that incorporates the advantages of paint style image editing programs with a simple and intuitive user interface to provide high functionality for editing document images.